Almost all the picture quality compression method widely used on the Internet, in digital cameras and elsewhere are irreversible compression method which efficiently reduce the volume of image data. Thanks to irreversible compression methods it has become easier to exchange and storage image data over networks.
Irreversible image compression methods include those like the JPEG compression method which permit flexible adjustment of the compression rate by setting compression parameters, while others achieve a higher compression rate at the cost of deterioration of picture quality or suppress deterioration of picture quality by lowering the compression rate.
Japanese Patent No. 2807222 is an example of an image compression device which sets compression parameters automatically to the optimum value. The image compression device disclosed here creates a residual image by subtracting between the original input image and the reconverted image. It then calculates the mean square error from data relating to this residual image, and uses this as a picture quality evaluation value to optimise compression parameters.
Another conventional technique for improving picture quality after decoding involves either compressing so that the whole pixel region is the same without regard to the characteristics of the display terminal etc., or as in the case of the image processing device disclosed in Japanese Laid-Open Patent Application H8[1996]-242376 taking the characteristics of the display terminal etc. into consideration and compensating by using a gamma curve to expand the dynamic range of pixel regions which are difficult to make out in detail on the display terminal etc.
An example of an image encoding device which uses different image compression methods according to pixel region in order to encode images is provided by the image data compression device disclosed in Japanese Laid-Open patent application H6[1994]-225160. The image data compression device disclosed here varies the image compression method depending on the number of colours contained within a specified region of the input image data. If a reversible compression method is used where the number of colours is great, the volume of compression data increased, and consequently it employs an irreversible compression method here, reserving the reversible method for where the number of colours is fewer.
However, although the aforesaid irreversible image compression method is capable of very flexible adjustments to picture quality and compression rate by means of compression parameters, its potential has not been used to the full. The principal reason for this is that even if the compression parameter has the same value, compression rate and picture quality after compression vary greatly, so that it is difficult to infer picture quality after compression from the compression parameters.
There have hitherto been devices which set compression parameters automatically, but they have employed mean square error or SN ratio as evaluation values, and have not sufficiently taken the characteristics of human vision into account. Accordingly, it has been necessary to set the compression parameters manually if the highest compression rate commensurate with picture quality acceptable to the user was needed, and it has been necessary to adjust compression parameters while checking picture quality after compression with the naked eye.
It is an object of the present invention to facilitate the automatic setting of compression parameters reflecting the characteristics of human vision by extracting pixel regions which are felt have deteriorated visually and either to utilise only the degree of distortion thereof for the evaluation of picture quality or by classifying the blocks according to their properties to set up a separate criterion for evaluation.
Especially with the JPEG compression method which compresses images by block units and other image compression methods utilising DCT conversion techniques, compression is achieved basically block by block, and there is no effect on picture quality between blocks after compression.
For this reason, the present invention proposes a method of evaluation whereby picture quality is assessed by calculating distortion by the block units used during compression, correctly detecting localised deterioration in picture quality.
Moreover, in image compression methods using DCT conversion techniques there has been a tendency when the conventionally used mean square error or SN ratio was applied to compressed picture quality for evaluation values to be small in relation to images with low activity of pixel values and greater where activity was higher.
Two well-known types of distortion peculiar to image compression using DCT conversion are block noise (noise whereby the periphery of a block becomes discontinuous) and mosquito noise (noise generated on the periphery of steep edges). Block noise occurs where the activity of pixel values is low, and as already mentioned, evaluation values with mean square error and SN ratio are small, making block noise difficult to detect with certainty.
Moreover, block noise tends to be prominent visually when it is flattened (when the AC component is 0).
In blocks where the AC component is 0 in both the X and Y directions all the pixel values within the block tally.
Moreover, in blocks where the AC component is all 0 in the X direction, pixel values tally within all the columns. Similarly, in blocks where the AC component is all 0 in the Y direction, pixel values tally within all the rows.
In the image encoding device to which the present invention pertains, blocks are classified according to the properties of pixel values within the block after compression, and it is possible to detect blocks in which noise is generated by establishing evaluation criteria which take into account the ease of noise generation within each classified block.
Blocks in which mosquito noise is generated have high conventional mean square error or SN ratio evaluation values. Blocks with high activity often have these high evaluation values even if mosquito noise is not generated, and it is impossible to detect mosquito noise with certainty.
Mosquito noise is a phenomenon where the effect of steep edges in some blocks appears on the periphery, and those blocks in which it is generated differ greatly as to distortion of pixels within the block.
The present invention makes it possible to detect mosquito noise correctly by calculating divergence in the differences between pixel values of the original image and compressed image.
There have hitherto been techniques for improving picture quality after decoding by compensatory expansion of the dynamic range of pixel regions which are difficult to make out in detail on the display terminal etc. by taking into account the properties of the display terminal etc. and using gamma curves. An example is the image processing device disclosed in Japanese Laid-Open Patent Application H8[1996]-242376. However, there have been no image encoding and decoding devices which encode after first reducing the dynamic range of the whole pixel value region where deterioration is not prominent thanks to the properties of the display terminal etc., reinstating the dynamic range of the whole decoded pixel value region.
It is an object of the image encoding device to which the present invention pertains to increase the compression rate considerably by encoding after first reducing the dynamic range of the whole pixel value region, for instance by reducing that of regions where the pixel value is low as in FIG. 8(A).
It is an object of the image decoding device to which the present invention pertains to expand the dynamic range of all the pixel value region of the decoded image so that after decoding it is the same as the whole pixel value region of the image prior to encoding, and by using it in combination with this image, decoding device to raise the compression rate considerably without any deterioration in the visual picture quality.
It is an object of the image encoding device to which the present invention pertains to place emphasis on an image compression method exhibiting high compression performance, being a combination of a plurality of image compression methods, by employing a method of evaluation which allows pronounced visual deterioration in the form of mosquito noise and block noise to be detected with certainty to allow regions where it is possible to secure a uniform level of picture quality set visually with the aid of a high-performance image compression all to be compressed by a high-performance image compression method, detecting other regions where it is possible to maintain picture quality in the same way with the aid of a secondary high-performance image compression method and compressing them with the aid of that image compression method, thus determining one after the other from the highest rate of compression so as to maintain a visually uniform level of picture quality and achieve the maximum compression rate.